ABSTRACT
BACKGROUND: Tools proposed to triage ED acuity in suspected COVID-19 were derived and validated in higher income settings during early waves of the pandemic. We estimated the accuracy of seven risk-stratification tools recommended to predict severe illness in the Western Cape, South Africa. METHODS: An observational cohort study using routinely collected data from EDs across the Western Cape, from 27 August 2020 to 11 March 2022, was conducted to assess the performance of the PRIEST (Pandemic Respiratory Infection Emergency System Triage) tool, NEWS2 (National Early Warning Score, version 2), TEWS (Triage Early Warning Score), the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS (Pandemic Medical Early Warning Score) in suspected COVID-19. The primary outcome was intubation or non-invasive ventilation, death or intensive care unit admission at 30 days. RESULTS: Of the 446 084 patients, 15 397 (3.45%, 95% CI 34% to 35.1%) experienced the primary outcome. Clinical decision-making for inpatient admission achieved a sensitivity of 0.77 (95% CI 0.76 to 0.78), specificity of 0.88 (95% CI 0.87 to 0.88) and the negative predictive value (NPV) of 0.99 (95% CI 0.99 to 0.99). NEWS2, PMEWS and PRIEST scores achieved good estimated discrimination (C-statistic 0.79 to 0.82) and identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.41 to 0.64. Use of the tools at recommended thresholds would have more than doubled admissions, with only a 0.01% reduction in false negative triage. CONCLUSION: No risk score outperformed existing clinical decision-making in determining the need for inpatient admission based on prediction of the primary outcome in this setting. Use of the PRIEST score at a threshold of one point higher than the previously recommended best approximated existing clinical accuracy.
Subject(s)
COVID-19 , Early Warning Score , Humans , Adult , Triage , COVID-19/diagnosis , Cohort Studies , Hospitalization , Retrospective StudiesABSTRACT
1482 Figure 2Performance of tools predicting composite primary outcome for the Omicron period[Figure omitted. See PDF] 1482 Table 1Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (entire study period)Tool N* C-statistic Threshold N (%) above threshold Sensitivity Specificity PPV NPV CRB-65 432,584 0.70 (0.70, 0.71) >0 102,964 (23.8%) 0.61 (0.61, 0.61) 0.78 (0.77, 0.78) 0.09 (0.09, 0.09) 0.98 (0.98, 0.98) NEWS2 433,101 0.80 (0.79, 0.80) >1 178835 (41.3%) 0.83 (0.83, 0.83) 0.6 (0.6,0.6) 0.07 (0.07–0.07) 0.99 (0.99, 0.99) PMEWS 438,810 0.79 (0.79, 0.79) >2 199,386 (45.4%) 0.85 (0.85, 0.85) 0.56 (0.56, 0.56) 0.06 (0.06, 0.07) 0.99 (0.99,0.99) PRIEST 438,880 0.82 (0.82, 0.82) >4 158,893 (36.2%) 0.83 (0.83, 0.83) 0.65 (0.65,0.66) 0.08 (0.08, 0.08) 0.99 (0.99, 0.99) WHO 437,850 0.71 (0.71, 0.72) >0 235,775 (53.8%) 0.82 (0.81, 0.82) 0.47 (0.47, 0.47) 0.05 (0.05, 0.05) 0.99 (0.99, 0.99) TEWS 432,612 0.72 (0.71, 0.72) >2 134,097 (31%) 0.62 (0.62, 0.62) 0.70 (0.70, 0.70) 0.07 (0.07, 0.07) 0.98 (0.98, 0.98) Quick COVID 446,088 0.70 (0.69, 0.70) >3 35,145 (7.9%) 0.33 (0.33, 0.33) 0.93 (0.93, 0.93) 0.14 (0.14, 0.14) 0.98 (0.98, 0.98) *Patients with <3 parameters were excluded from analysis when estimating performance 1482 Table 2Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (Omicron period)Tool N* C-statistic Threshold N (%) above threshold Sensitivity Specificity PPV NPV CRB-65 136,961 0.69 (0.68, 0.70) >0 31,373 (22.9%) 0.59 (0.59, 0.59) 0.78 (0.78, 0.78) 0.05 (0.05, 0.05) 0.99 (0.99, 0.99) NEWS2 137,125 0.77 (0.76, 0.78) >1 76,183 (55.6%) 0.87 (0.87, 0.87) 0.45 (0.45, 0.45) 0.03 (0.03, 0.03) 0.99 (0.99, 0.99) PMEWS 138,954 0.76 (0.75, 0.76) >2 59,876 (43.1%) 0.80 (0.80, 0.80) 0.58 (0.58, 0.58) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) PRIEST 158,893 0.78 (0.77, 0.79) >4 46,529 (33.5%) 0.75 (0.75, 0.75) 0.67 (0.67, 0.67) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) WHO 138,666 0.62 (0.61, 0.63) >0 72,599 (52.4%) 0.70 (0.70, 0.70) 0.48 (0.48, 0.48) 0.03 (0.03, 0.03) 0.99 (0.99, 0.99) TEWS 136,967 0.73 (0.72, 0.74) >2 39,509 (28.8%) 0.64 (0.64, 0.64) 0.72 (0.72, 0.72) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) Quick COVID 140520 0.61 (0.60, 0.63) >3 8,210 (6.4%) 0.17 (0.17, 0.17) 0.94 (0.94, 0.94) 0.06 (0.06, 0.06) 0.98 (0.98, 0.98) *Patients with <3 parameters were excluded from analysis when estimating performanceResults and ConclusionOf the 446,084 patients, 15,397 patients (3.45%, 95% CI:34% to 35.1%) experienced the primary outcome. Figure 1 presents the ROC curves for the triage tools for the total study period and figure 2 for the period of the Omicron wave. NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.47 (NEWS2) o 0.65 (PRIEST tool). The low prevalence of the primary outcome, especially in the Omicron period, meant use of these tools would have more than doubled admissions with only a small reduction in risk of false negative triage.Triage tools developed specifically in low- and middle-income settings may be needed to provide accurate risk prediction. Existing triage tools may need to be used at varying thresholds to reflect different baseline-line risks of adverse outcomes in different settings.